{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Using ZhipuAI Embedding API to get the embedding of a text\n",
    "\n",
    "**This tutorial is available in English and is attached below the Chinese explanation**\n",
    "\n",
    "本代码旨在使用 ZhipuAI 提供的 embedding-2 模型，对一段文本进行 Embedding。此代码包含了以下内容：\n",
    "1. 使用 SDK简单调用 ZhipuAI 的 Embedding API\n",
    "2. 使用向量数据库和 Embedding 进行检索\n",
    "\n",
    "This cookbook is designed to use the embedding-2 model provided by ZhipuAI to embed a piece of text. This code contains the following content:\n",
    "1. Use SDK to simply call ZhipuAI’s Embedding API\n",
    "2. Retrieve using vector database and Embedding"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "7ab0b5c8bba9b509"
  },
  {
   "cell_type": "markdown",
   "source": [
    "运行本代码之前，你需要安装一些额外的依赖\n",
    "\n",
    "Before running this code, you need to install some additional dependencies"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "f11fff64e466b2a9"
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: faiss-cpu in /Users/zr/Code/glm-cookbook/venv/lib/python3.9/site-packages (1.7.4)\r\n",
      "Requirement already satisfied: scikit-learn in /Users/zr/Code/glm-cookbook/venv/lib/python3.9/site-packages (1.4.0)\r\n",
      "Requirement already satisfied: scipy in /Users/zr/Code/glm-cookbook/venv/lib/python3.9/site-packages (1.12.0)\r\n",
      "Requirement already satisfied: numpy<2.0,>=1.19.5 in /Users/zr/Code/glm-cookbook/venv/lib/python3.9/site-packages (from scikit-learn) (1.26.3)\r\n",
      "Requirement already satisfied: joblib>=1.2.0 in /Users/zr/Code/glm-cookbook/venv/lib/python3.9/site-packages (from scikit-learn) (1.3.2)\r\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in /Users/zr/Code/glm-cookbook/venv/lib/python3.9/site-packages (from scikit-learn) (3.2.0)\r\n",
      "\u001B[33mDEPRECATION: textract 1.6.5 has a non-standard dependency specifier extract-msg<=0.29.*. pip 24.0 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of textract or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063\u001B[0m\u001B[33m\r\n",
      "\u001B[0m"
     ]
    }
   ],
   "source": [
    "!pip install faiss-cpu scikit-learn scipy"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-01-28T04:53:46.331540Z",
     "start_time": "2024-01-28T04:53:44.952464Z"
    }
   },
   "id": "3fc56e392faf01f8"
  },
  {
   "cell_type": "markdown",
   "source": [
    "接着，设置好 API KEY，然后就可以开始了。\n",
    "\n",
    "Next, set the API KEY and you’re ready to go."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "f328002422d78027"
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "outputs": [],
   "source": [
    "import os\n",
    "from zhipuai import ZhipuAI\n",
    "\n",
    "os.environ[\"ZHIPUAI_API_KEY\"] = \"your api key\"\n",
    "client = ZhipuAI()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-01-28T04:53:46.349431Z",
     "start_time": "2024-01-28T04:53:46.334611Z"
    }
   },
   "id": "a8212c95e046754c"
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Simple Embedding Sdk Usage\n",
    "我们先设置一段用于检索的文本,我们需要将这段文本想量化，这里，我们演示了最基础的调用方式，直接对整个文本进行想量化。\n",
    "\n",
    "We first set up a piece of text for retrieval. We need to quantify this text. Here, we demonstrate the most basic calling method to directly quantify the entire text."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "3d73405de41786fe"
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [],
   "source": [
    "embedding_text = \"hello world\"\n",
    "response = client.embeddings.create(\n",
    "    model=\"embedding-2\",\n",
    "    input=embedding_text,\n",
    ")"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-01-28T04:53:47.306124Z",
     "start_time": "2024-01-28T04:53:46.351187Z"
    }
   },
   "id": "260a56bccff0f82a"
  },
  {
   "cell_type": "markdown",
   "source": [
    "返回的结果将会包含两组信息，分别是\n",
    "1. 使用的 token 数量，位于 `response.usage.total_tokens` 中。\n",
    "2. Embedding 后的向量，是一个 `1024` 维的向量，位于 `response.data[0].embedding` 中。\n",
    "\n",
    "The results returned will contain instant information, namely\n",
    "1. The number of tokens used, located in `response.usage.total_tokens`.\n",
    "2. The support behind Embedding is a `1024` dimension support, located in `response.data[0].embedding`.\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "98428e6d1a49da54"
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "outputs": [
    {
     "data": {
      "text/plain": "3"
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.usage.total_tokens"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-01-28T04:53:47.319802Z",
     "start_time": "2024-01-28T04:53:47.306802Z"
    }
   },
   "id": "55fe1d03617043bd"
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "outputs": [
    {
     "data": {
      "text/plain": "[-0.0686611458659172,\n -0.013444190844893456,\n -0.018471742048859596,\n 0.022484764456748962,\n 0.019037242978811264,\n -0.031042901799082756,\n -0.02871578000485897,\n 0.04731578379869461,\n -0.02328941971063614,\n -0.001094392384402454,\n 0.006920468993484974,\n 0.009118706919252872,\n 0.0023793329019099474,\n 0.038529783487319946,\n 0.054401643574237823,\n -0.006751869339495897,\n 0.013510574586689472,\n 0.010793854482471943,\n -0.002416919684037566,\n -0.04071984812617302,\n -0.026856202632188797,\n -0.00982305034995079,\n -0.00045497293467633426,\n -0.06797515600919724,\n 0.022623205557465553,\n -0.03774299845099449,\n -0.01211058720946312,\n -0.03959469869732857,\n -0.005243128631263971,\n -0.05262289568781853,\n 0.07098551094532013,\n 0.01295552123337984,\n 0.005307368002831936,\n -0.0066171977669000626,\n 0.020235709846019745,\n 0.03266848251223564,\n 0.0027419289108365774,\n 0.01981448009610176,\n 0.010425595566630363,\n -0.013362426310777664,\n -0.002935317112132907,\n 0.008259180933237076,\n 0.039446763694286346,\n -0.018463339656591415,\n 0.03158793970942497,\n 0.02589007280766964,\n -0.013421733863651752,\n 0.02541954070329666,\n -0.04310854524374008,\n -0.04341774433851242,\n -0.0029312949627637863,\n 0.012374965474009514,\n -0.0016316340770572424,\n 0.06733269989490509,\n -0.020467590540647507,\n -0.006029242649674416,\n -0.040173765271902084,\n 0.033218856900930405,\n -0.0002335157332709059,\n 0.011129679158329964,\n -0.016135357320308685,\n 0.019016027450561523,\n -0.01691068522632122,\n -0.0029231819789856672,\n 0.02368856593966484,\n -0.010169661603868008,\n 0.018211055546998978,\n 0.017406824976205826,\n 0.013354972936213017,\n 0.01426671538501978,\n 0.01752866804599762,\n -0.00489907106384635,\n 0.016116749495267868,\n 0.006970027927309275,\n 0.007812018506228924,\n 0.005871908273547888,\n -0.034876320511102676,\n 0.031976524740457535,\n 0.010464166291058064,\n -0.005135721061378717,\n -0.058190979063510895,\n 0.06869885325431824,\n 0.010511939413845539,\n -0.058122418820858,\n 0.013930689543485641,\n 0.0036941387224942446,\n 0.02603772096335888,\n 0.01923133060336113,\n 0.0287312064319849,\n -0.003150025149807334,\n -0.03144118934869766,\n 0.054575007408857346,\n 0.0008975350065156817,\n -0.01382535696029663,\n -0.02995622530579567,\n -0.00703388312831521,\n 0.09038784354925156,\n -0.02795783244073391,\n 0.03960176557302475,\n -0.01950475387275219,\n 0.036432381719350815,\n -0.010986797511577606,\n -0.06299294531345367,\n -0.026158133521676064,\n -0.024626722559332848,\n 0.016671055927872658,\n -0.004099599551409483,\n -0.008835511282086372,\n 0.04790937155485153,\n -0.08248146623373032,\n 0.026137979701161385,\n -0.033169444650411606,\n -0.01324823871254921,\n 0.039906878024339676,\n -0.024698765948414803,\n -0.01900755614042282,\n 0.00417142640799284,\n 0.055199120193719864,\n -0.011133449152112007,\n -0.019527850672602654,\n -0.020882338285446167,\n 0.05726376175880432,\n 0.036596864461898804,\n -0.01725693792104721,\n 0.009579005651175976,\n -0.0077188340947031975,\n -0.005907496437430382,\n -0.04102674871683121,\n -0.013689271174371243,\n -0.024776456877589226,\n 0.04155110567808151,\n -0.005760093219578266,\n 0.029996512457728386,\n 0.035902153700590134,\n -0.01854644902050495,\n 0.018265273422002792,\n -0.02270532213151455,\n 0.012295113876461983,\n -0.029534807428717613,\n -0.011146875098347664,\n 0.03911413624882698,\n 0.03445799648761749,\n 0.0026923262048512697,\n -0.02497350051999092,\n -0.002484184456989169,\n 0.029298922047019005,\n 0.015868470072746277,\n -0.01717495359480381,\n 0.009124355390667915,\n 0.023121168836951256,\n 0.03528181090950966,\n 0.04876340180635452,\n 6.733599680446012e-13,\n 0.00115889310836792,\n 0.008062893524765968,\n -0.008113579824566841,\n -0.021783573552966118,\n 0.019610436633229256,\n -0.022788485512137413,\n -0.029813099652528763,\n 0.004610877484083176,\n 0.053926557302474976,\n 0.0005953312502242625,\n 6.7971850436774e-18,\n -0.080357626080513,\n 0.029611634090542793,\n -0.010597563348710537,\n -0.021079029887914658,\n 0.05958663672208786,\n -0.03219643980264664,\n 0.05495787411928177,\n -0.013343187980353832,\n -0.006230346858501434,\n 0.011531505733728409,\n 0.005264536943286657,\n -0.023715991526842117,\n 0.0494309663772583,\n -0.007404175121337175,\n -0.02102973870933056,\n -0.016198666766285896,\n 0.020310278981924057,\n -0.013366582803428173,\n -0.009700734168291092,\n -0.007022622972726822,\n -0.004189822357147932,\n -0.012449205853044987,\n 0.036201201379299164,\n -0.022187702357769012,\n -0.015810534358024597,\n 0.03444365784525871,\n -0.022403063252568245,\n -0.02215494029223919,\n -0.051733966916799545,\n 0.005821939092129469,\n 0.04652797058224678,\n 0.012784986756742,\n 0.07014822214841843,\n 0.013883877545595169,\n 0.01817825436592102,\n -0.015504635870456696,\n -0.00692959688603878,\n -0.04102010279893875,\n 0.0036039059050381184,\n -0.027148496359586716,\n 0.06218138337135315,\n -0.026437943801283836,\n 0.0012272725580260158,\n -0.013544419780373573,\n -0.02053963579237461,\n -0.02564742974936962,\n -0.06336929649114609,\n 0.009544135071337223,\n 0.006105345673859119,\n 0.015609541907906532,\n 0.02643589675426483,\n 0.02901626005768776,\n 0.02781856805086136,\n 0.004972506780177355,\n 0.028753850609064102,\n -0.02549208700656891,\n -0.027011102065443993,\n -0.007183559704571962,\n 0.04963209852576256,\n -0.019284840673208237,\n 0.061628542840480804,\n -0.06947430223226547,\n 0.014986409805715084,\n 0.06629189848899841,\n 0.03901299834251404,\n 0.037245552986860275,\n -0.007003523875027895,\n 0.003208194626495242,\n 0.00632699066773057,\n 0.07249391078948975,\n -0.018126990646123886,\n -0.010810993611812592,\n -0.03462694585323334,\n 0.030121825635433197,\n -0.006192164495587349,\n 0.0207488052546978,\n 0.0004084862011950463,\n 0.0021521716844290495,\n -0.07723154872655869,\n -0.027105292305350304,\n 0.09299276024103165,\n -0.008386681787669659,\n -0.02482505515217781,\n 0.013356814160943031,\n -0.006523450370877981,\n 0.025980668142437935,\n 0.002884193090721965,\n -0.023433178663253784,\n -0.009167872369289398,\n -0.047511953860521317,\n -0.009951296262443066,\n 0.045603055506944656,\n -0.017056019976735115,\n 0.029130468145012856,\n 0.006366269662976265,\n 0.036938637495040894,\n 0.011763624846935272,\n -0.003082566661760211,\n 0.016705602407455444,\n -0.01237360667437315,\n 0.049224063754081726,\n -0.004210638348013163,\n 0.049029652029275894,\n 0.052271366119384766,\n 0.02389197237789631,\n -0.026023665443062782,\n 0.027766546234488487,\n -0.041155364364385605,\n 0.01640625298023224,\n -0.02951931394636631,\n 0.005665000062435865,\n 0.02422277256846428,\n -0.01977425068616867,\n -0.028980063274502754,\n -0.012119846418499947,\n 0.010823347605764866,\n 0.03843553736805916,\n 0.04263130575418472,\n -0.042777858674526215,\n 0.04571867734193802,\n -0.07468520849943161,\n 0.02895752713084221,\n 0.03766751289367676,\n -0.011985254473984241,\n -0.030248666182160378,\n 8.731557318242267e-05,\n -0.04384244233369827,\n -0.009878401644527912,\n -0.02282634936273098,\n 0.05844591557979584,\n -0.029653145000338554,\n -0.026085788384079933,\n -0.022633815184235573,\n 0.00936113577336073,\n 0.03708799183368683,\n -0.04872399941086769,\n -0.0031828952487558126,\n -0.030537912622094154,\n 0.024798814207315445,\n -0.05244946479797363,\n 0.0003625735407695174,\n -0.031238922849297523,\n 0.009864666499197483,\n 0.030254719778895378,\n -0.018525172024965286,\n 0.0365760400891304,\n 0.03824739530682564,\n 0.011297459714114666,\n 0.015852803364396095,\n -0.0024812270421534777,\n -0.0022923597134649754,\n -0.04799686372280121,\n 0.00263436371460557,\n 0.03673126548528671,\n 0.027995776385068893,\n 0.0005418980726972222,\n -0.020662782713770866,\n -0.005910716950893402,\n 0.03461920842528343,\n -0.01910475268959999,\n 0.032397717237472534,\n 0.0043220845982432365,\n -0.03155035153031349,\n 0.0020361952483654022,\n -0.05774731561541557,\n -0.0035276617854833603,\n -0.011699585244059563,\n 0.0025115334428846836,\n -0.005223398096859455,\n -0.019116053357720375,\n -0.06632909178733826,\n -0.04017031192779541,\n -0.09462197124958038,\n -0.022814011201262474,\n -0.02082228474318981,\n -0.016461731866002083,\n 0.005579950753599405,\n -0.0549478754401207,\n 0.003619466908276081,\n 0.03709429129958153,\n 0.01916579343378544,\n 0.02977430634200573,\n 0.06731945276260376,\n -0.06556983292102814,\n 0.042431727051734924,\n 0.05668016895651817,\n -0.01173924095928669,\n -0.026752976700663567,\n 0.057874467223882675,\n -0.018476461991667747,\n 0.06902920454740524,\n -0.0039050662890076637,\n 0.009448741562664509,\n -0.03824847564101219,\n 0.040863633155822754,\n 0.032791849225759506,\n 0.0823778435587883,\n 0.020827198401093483,\n 0.019535817205905914,\n 0.057014402002096176,\n -0.020355960354208946,\n -0.022413665428757668,\n -0.012888146564364433,\n 0.022481802850961685,\n -0.008867125026881695,\n 0.01681927964091301,\n 0.00878897588700056,\n -0.051628805696964264,\n 0.002621337305754423,\n -0.01074040587991476,\n -0.017983799800276756,\n 0.004298462066799402,\n 0.0063759260810911655,\n -0.06025298312306404,\n 0.05281820148229599,\n -0.04640582203865051,\n 0.0017318835016340017,\n -0.020794685930013657,\n 0.024691691622138023,\n 0.033097218722105026,\n -0.026628997176885605,\n -0.05622910335659981,\n 0.008220401592552662,\n 0.05035203695297241,\n 0.00543695455417037,\n -0.006103424355387688,\n 0.02428724244236946,\n -0.029122503474354744,\n -0.014743771404027939,\n -0.007732127793133259,\n -0.044956814497709274,\n 0.014716319739818573,\n -0.0012390483170747757,\n 0.04239977151155472,\n -0.004524266347289085,\n 0.0035071622114628553,\n -0.07657907903194427,\n 0.07588154077529907,\n -0.07263141870498657,\n 0.0018646480748429894,\n -0.034290678799152374,\n -0.025697976350784302,\n 0.01793980970978737,\n -0.015341035090386868,\n 0.018766721710562706,\n 0.04571676626801491,\n -0.010275262407958508,\n -0.005144188646227121,\n -0.007429498713463545,\n -0.003941146656870842,\n 0.030038483440876007,\n 0.0255044624209404,\n 0.02979501709342003,\n -0.0037338382098823786,\n -0.0034150583669543266,\n -0.03904471546411514,\n 0.012804191559553146,\n 0.043393295258283615,\n 0.034600816667079926,\n 0.016258500516414642,\n 0.005099343601614237,\n 0.017060061916708946,\n -0.006512341555207968,\n -0.005507213994860649,\n -0.010651683434844017,\n -0.02395104430615902,\n 0.002539873355999589,\n 0.0383041687309742,\n 0.012655039317905903,\n 0.03687077760696411,\n 0.03140077739953995,\n -0.005944365169852972,\n -0.0937890112400055,\n 0.001719155814498663,\n 0.0160214900970459,\n 0.009523943066596985,\n -0.03938332945108414,\n -0.04948505759239197,\n -0.010117987170815468,\n -0.08796310424804688,\n 0.030439626425504684,\n -0.02960863709449768,\n -0.04932667687535286,\n -0.02016935497522354,\n -0.04892067611217499,\n -0.04260357841849327,\n 0.04945063218474388,\n 0.04703057184815407,\n -0.012691342271864414,\n 0.023807445541024208,\n 0.04590703547000885,\n 0.03718017414212227,\n -0.014847848564386368,\n -0.04076298698782921,\n 0.0525132454931736,\n 0.03648122027516365,\n 0.023306304588913918,\n 0.0034071116242557764,\n 0.030335964635014534,\n -0.028198739513754845,\n -0.0026151309721171856,\n -0.05414324998855591,\n 0.024970894679427147,\n 0.02116912230849266,\n 0.05650952458381653,\n 0.01367255300283432,\n 0.04281512647867203,\n -0.017477918416261673,\n 0.06370220333337784,\n -0.030366050079464912,\n -0.01503436267375946,\n 0.06775175034999847,\n -0.0571473203599453,\n 0.01651705801486969,\n 0.047687578946352005,\n -0.033661138266325,\n 0.01982000283896923,\n -0.016214339062571526,\n 0.011859903112053871,\n 0.0027479729615151882,\n 0.005980032961815596,\n 0.011898020282387733,\n 0.0033503554295748472,\n -0.005240658298134804,\n -0.04294901713728905,\n 0.011225125752389431,\n 0.11617445945739746,\n 0.023818043991923332,\n -0.0319170206785202,\n 0.03989262878894806,\n 0.0031098437029868364,\n -0.02383420616388321,\n -0.014400104060769081,\n -0.009080734103918076,\n 0.03199436143040657,\n 0.010127007961273193,\n 0.03671049699187279,\n 0.02991914376616478,\n 0.015422896482050419,\n 0.04503192752599716,\n 0.026328664273023605,\n -0.0002394843177171424,\n 0.019833697006106377,\n -0.041290730237960815,\n 0.017879674211144447,\n -0.009778359904885292,\n -0.06935691833496094,\n -0.002290479140356183,\n 0.01271056942641735,\n -0.023081354796886444,\n -0.020096899941563606,\n 0.05092058703303337,\n 0.03828210383653641,\n 0.0303825531154871,\n -0.05565861985087395,\n -0.0067252712324261665,\n -0.010272965766489506,\n -0.04243774339556694,\n 0.0024574638810008764,\n -0.0035430530551820993,\n -0.02610067091882229,\n -0.04106161743402481,\n -0.034292712807655334,\n 0.05186228081583977,\n 0.008207899518311024,\n -0.0038809773977845907,\n 0.009965592995285988,\n -0.019494015723466873,\n 0.023449240252375603,\n -0.014485231600701809,\n -0.05345160886645317,\n -0.025405988097190857,\n 0.014611386694014072,\n 0.030365128070116043,\n -0.009524110704660416,\n -0.031987227499485016,\n -0.04578404873609543,\n 0.063722163438797,\n 0.05397564545273781,\n -0.007077388931065798,\n 0.014727489091455936,\n -0.028178390115499496,\n 0.019117658957839012,\n -0.010502875782549381,\n -0.010865448974072933,\n -0.052358478307724,\n -0.0042685335502028465,\n 0.00947460439056158,\n -0.03894397243857384,\n -0.012017674744129181,\n -0.017973652109503746,\n -0.03544618934392929,\n 0.030620725825428963,\n -0.01907806657254696,\n 0.040120698511600494,\n 0.0026760324835777283,\n -0.026832466945052147,\n -0.009400846436619759,\n 0.03985665738582611,\n 0.03585894778370857,\n -0.021241474896669388,\n -0.03968792408704758,\n -0.004903949331492186,\n 0.00940778385847807,\n 0.006126279477030039,\n 0.0092139458283782,\n 0.0042695701122283936,\n 0.04420691356062889,\n -0.027922043576836586,\n -0.005955851636826992,\n -0.002934196265414357,\n -0.008996888063848019,\n 0.015316864475607872,\n 0.023877961561083794,\n -0.025020968168973923,\n -0.016880379989743233,\n 0.018151678144931793,\n 0.017647743225097656,\n -0.062238745391368866,\n 0.03248075395822525,\n 0.006667007692158222,\n -0.04739508405327797,\n 0.004768576007336378,\n 0.035627346485853195,\n 0.02130303904414177,\n -0.039423201233148575,\n -0.0116564417257905,\n -0.05120363086462021,\n 0.01264998409897089,\n 0.014586124569177628,\n 0.05561336129903793,\n 0.02678016573190689,\n 0.007879606448113918,\n 0.03050336427986622,\n 0.029476992785930634,\n -0.018447570502758026,\n 0.02819851040840149,\n 0.02149323560297489,\n 0.032570112496614456,\n -0.0026097504887729883,\n 0.02346055582165718,\n 0.056656818836927414,\n 0.0012466296320781112,\n -0.020395368337631226,\n -0.04610488936305046,\n -0.04349490627646446,\n 0.017855513840913773,\n 0.024863265454769135,\n -0.004241878166794777,\n -0.0046417624689638615,\n 0.04547155275940895,\n -0.02706187590956688,\n 0.02845567837357521,\n 0.040567196905612946,\n 0.004537360277026892,\n 0.0006889966898597777,\n 0.025905942544341087,\n 0.0610479936003685,\n 0.007570349611341953,\n -0.04003802314400673,\n -0.025124015286564827,\n 0.017972610890865326,\n 0.023247895762324333,\n 0.001686158124357462,\n 0.030708372592926025,\n 0.000995647395029664,\n 0.024648044258356094,\n 0.039076730608940125,\n 0.016621317714452744,\n 0.0170854851603508,\n -0.03799745440483093,\n -0.019697019830346107,\n 0.05935932323336601,\n 0.00015069573419168591,\n -0.008740485645830631,\n -0.003180120373144746,\n 0.045882537961006165,\n -0.018369685858488083,\n -0.06390439718961716,\n -0.0014109197072684765,\n -0.02612662874162197,\n -0.004363558255136013,\n -0.03147212788462639,\n 0.009026444517076015,\n -0.010284069925546646,\n -0.0535765066742897,\n -0.003603526158258319,\n 0.027936870232224464,\n 0.02868816815316677,\n -0.012753469869494438,\n -0.03481282666325569,\n 0.05918850749731064,\n -0.04248080775141716,\n 0.04014042392373085,\n -0.032455164939165115,\n -0.00944066047668457,\n 0.022045424208045006,\n -0.0012602821225300431,\n 0.007151828613132238,\n 0.002689308486878872,\n -0.025212984532117844,\n -0.06023767217993736,\n -0.009374289773404598,\n -0.010336579754948616,\n 0.016192832961678505,\n -0.02552781067788601,\n -0.03459396958351135,\n 0.0023592000361531973,\n 0.01923399604856968,\n -0.03923024609684944,\n 0.012079686857759953,\n -0.01006272528320551,\n 0.009061437100172043,\n 0.027686525136232376,\n -0.007527717389166355,\n -0.02419736050069332,\n 0.003432651748880744,\n 0.01383696123957634,\n 0.025039497762918472,\n 0.0009954270208254457,\n 0.00033183599589392543,\n -0.00491346837952733,\n -0.01023690216243267,\n 0.009046546183526516,\n 0.006139616947621107,\n -0.010863746516406536,\n 0.02833583392202854,\n 0.0002942250284831971,\n 0.03188852593302727,\n 0.026148105040192604,\n 0.0027173522394150496,\n 0.0009262695675715804,\n 0.0005940152332186699,\n 0.03222028538584709,\n -0.007064270321279764,\n 0.010001598857343197,\n 0.011630852706730366,\n 0.001634002081118524,\n 0.0027655314188450575,\n 0.03972821310162544,\n 0.006402665749192238,\n -0.052644047886133194,\n 0.030599258840084076,\n -0.00866185687482357,\n -0.014802517369389534,\n 0.009288080967962742,\n 0.07548633962869644,\n 0.014663483016192913,\n 0.003432302502915263,\n 0.022487709298729897,\n 0.007726288866251707,\n 0.0062892623245716095,\n -0.01793493889272213,\n -0.008070288226008415,\n -0.07193630188703537,\n -0.020588019862771034,\n -0.05353610962629318,\n 0.02655038796365261,\n -0.032728780061006546,\n 0.055529478937387466,\n -0.004176323302090168,\n 0.01328854076564312,\n 0.02623140439391136,\n 0.025355705991387367,\n -0.02338414639234543,\n -0.01097083929926157,\n -0.007806648965924978,\n 0.008555931970477104,\n 0.025737762451171875,\n -0.0028544762171804905,\n 0.033403534442186356,\n -0.007280995137989521,\n 0.0017917221412062645,\n -0.03242639824748039,\n 0.028888322412967682,\n -0.018810195848345757,\n -0.059775691479444504,\n 0.005400741007179022,\n 0.03233535215258598,\n -0.039553094655275345,\n -0.012187868356704712,\n -0.005811396986246109,\n 0.02045256830751896,\n -0.03556112200021744,\n -0.018132729455828667,\n -0.025311820209026337,\n -0.021036721765995026,\n -0.0003649243153631687,\n -0.04367770999670029,\n 0.014556805603206158,\n -0.00761281605809927,\n 0.027278248220682144,\n 0.024618202820420265,\n -0.0394587367773056,\n 0.04471287876367569,\n -0.0378715842962265,\n 0.02530796080827713,\n 0.0004432304122019559,\n -0.005936204921454191,\n -0.001937714172527194,\n -0.021863821893930435,\n -0.010067571885883808,\n -0.02945118397474289,\n 0.0070449900813400745,\n 0.027633553370833397,\n -0.038484130054712296,\n -0.008039632812142372,\n 0.011794988065958023,\n 0.0035554615315049887,\n 0.05512082949280739,\n 0.0216682031750679,\n -0.026396218687295914,\n 0.05494032800197601,\n 0.029046112671494484,\n 0.004082652274519205,\n 0.004158891271799803,\n -0.00017191491497214884,\n -0.022983325645327568,\n -0.013886348344385624,\n 0.010774831287562847,\n -0.03276075795292854,\n 0.042861588299274445,\n 0.013082452118396759,\n 0.008702628314495087,\n -0.026942940428853035,\n 0.019098922610282898,\n -0.05201485753059387,\n -0.034426286816596985,\n 0.014977381564676762,\n 0.019130486994981766,\n -0.01520777028053999,\n -0.006143367383629084,\n 0.008266654796898365,\n 0.00032013270538300276,\n 0.03689330071210861,\n -0.03954186663031578,\n 8.639750026517845e-10,\n -0.029615076258778572,\n 0.0019216021755710244,\n 0.006071808747947216,\n 0.04944751039147377,\n -0.001918799476698041,\n -0.06097138673067093,\n 0.04775973781943321,\n 0.006151828449219465,\n -0.027137964963912964,\n 0.0036747981794178486,\n -0.022765988484025,\n -0.04734524339437485,\n 0.07009483873844147,\n 0.0022946924436837435,\n -0.04206188768148422,\n 0.027664698660373688,\n 0.036499228328466415,\n -0.004192046821117401,\n -0.041403789073228836,\n -0.04294484108686447,\n 0.04920492693781853,\n -0.06851523369550705,\n 0.023791395127773285,\n 0.010006009601056576,\n 0.03783891350030899,\n 0.06313171237707138,\n 0.0002179962320951745,\n 0.031961627304553986,\n 0.0013832165859639645,\n 0.004599149338901043,\n 0.005144564434885979,\n 0.021992094814777374,\n -0.031400639563798904,\n 0.08426246792078018,\n 0.0326601006090641,\n -0.020005732774734497,\n -0.033078402280807495,\n -0.012945818714797497,\n -0.03115066885948181,\n 0.024845466017723083,\n 0.012810704298317432,\n -0.03880108520388603,\n 0.06370536237955093,\n 0.011302286759018898,\n -0.005743459798395634,\n -0.030280357226729393,\n -0.012708024121820927,\n -0.011231999844312668,\n -0.006570795550942421,\n 0.010787793435156345,\n -0.050946786999702454,\n 0.021102603524923325,\n 0.012825208716094494,\n -0.046950384974479675,\n -0.013229338452219963,\n 0.04810425639152527,\n -0.005614378955215216,\n 0.01618902198970318,\n 0.0050149355083703995,\n -0.011004256084561348,\n -0.012941168621182442,\n 0.06442274898290634,\n 0.0797533467411995,\n -0.045380719006061554,\n 0.022128824144601822,\n -0.0010145255364477634,\n 0.030595919117331505,\n 0.007922482676804066,\n 0.0048902626149356365,\n 0.015477096661925316,\n -0.0300014466047287,\n 0.02165839821100235,\n 0.0816333070397377,\n 0.015771659091114998,\n 0.08461878448724747,\n 0.03810809552669525,\n -0.023495236411690712,\n -0.03600459545850754,\n 0.013399004936218262,\n -0.006224536336958408,\n -0.017323624342679977,\n 0.0024117683060467243,\n -0.012019157409667969,\n -0.022454502061009407,\n -0.04534297063946724,\n -0.030974872410297394,\n -0.02765030786395073,\n -0.0147556746378541,\n 0.006648638751357794,\n -0.0025174580514431,\n -0.04558633267879486,\n 0.003337785368785262,\n -0.013451487757265568,\n 0.04640134796500206,\n -0.02271147258579731,\n 0.02599620446562767,\n -0.010944562032818794,\n -0.013562955893576145,\n 0.010096298530697823,\n 0.03230397403240204,\n -0.030312975868582726,\n 0.028250718489289284,\n -3.4810344039343535e-27,\n -0.014488795772194862,\n 0.002646281151100993,\n 0.04128938913345337,\n -0.030297299847006798,\n 0.0627698302268982,\n 0.04880394786596298,\n 0.034271035343408585,\n 0.009493008255958557,\n -0.013205168768763542,\n -0.027096139267086983,\n 0.024005591869354248,\n -0.05825798213481903,\n 0.05053164064884186,\n -0.011485220864415169,\n -0.013136083260178566,\n 0.032413795590400696,\n 0.027597393840551376,\n 0.03995421528816223,\n -0.016267143189907074,\n 0.017166374251246452,\n -0.020372195169329643,\n -0.036872245371341705,\n 0.001354234293103218,\n 0.024616308510303497,\n -0.028674805536866188,\n 0.016466811299324036,\n -0.026398055255413055,\n 0.04596923664212227,\n 8.075759978964925e-05,\n -0.00038344308268278837,\n 0.018210897222161293,\n 0.003578722709789872,\n 0.011779277585446835,\n 0.012888804078102112,\n -0.0017411574954167008,\n -0.08686424046754837,\n -0.0035689491778612137,\n 0.015542675741016865,\n 0.016847986727952957,\n 0.008972247131168842,\n -0.0009257740457542241,\n 0.015049522742629051,\n -0.0714339166879654,\n -0.045495085418224335,\n 0.01237107627093792,\n -0.03101545199751854,\n 0.037334151566028595,\n -0.005300915800035,\n -0.007253642193973064,\n -0.0080726882442832,\n 0.06615889817476273,\n -0.026680726557970047,\n -0.03871295228600502,\n -0.05425245687365532,\n 0.003784582717344165,\n 0.009227234870195389,\n 0.03633220121264458,\n -0.0201196800917387,\n -0.014355956576764584,\n -0.007434249855577946,\n 0.04193112626671791,\n -0.037027060985565186,\n 0.025189176201820374,\n -0.012325356714427471,\n -0.013097268529236317,\n 0.005645446944981813,\n 0.036092180758714676,\n -0.023037100210785866,\n -0.04440903291106224,\n 0.03967369720339775,\n -0.0010351624805480242,\n -0.010031132027506828,\n 0.01302633248269558,\n 0.011393750086426735,\n 0.018527178093791008,\n -0.013136240653693676,\n -0.038354597985744476,\n -0.08610757440328598,\n 0.016406746581196785,\n 0.02782866172492504,\n 0.0240290816873312,\n 0.04056093096733093,\n -0.030317967757582664,\n -0.015508404932916164,\n -0.020976359024643898,\n -0.030543819069862366,\n 0.02820868231356144,\n 0.01985303685069084,\n 0.06163209304213524,\n 0.005645573604851961,\n -0.041097357869148254,\n 0.046946704387664795,\n 0.04446009546518326,\n 0.023807251825928688,\n 0.013727196492254734,\n -0.017404528334736824,\n ...]"
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.data[0].embedding"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-01-28T04:53:47.330275Z",
     "start_time": "2024-01-28T04:53:47.326376Z"
    }
   },
   "id": "1a15ad3e016252c0"
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Use vector database and search\n",
    "\n",
    "我们可以开始尝试一些更高级的操作。例如对文章进行分词后存入到数据库。我们选取了来自[AGENT AI:\n",
    "SURVEYING THE HORIZONS OF MULTIMODAL INTERACTION](https://arxiv.org/abs/2401.03568)的部分文章段落并进行嵌入。\n",
    "由于文章太长，我们先要对文章进行分词。在这里，我们使用没有任何优化的顺序分词器，将文章分成了 150 个字符一段的小文本块。\n",
    "\n",
    "We can start trying some more advanced operations. For example, the article is segmented and stored in the database. We selected a selection from [AGENT AI:\n",
    "SURVEYING THE HORIZONS OF MULTIMODAL INTERACTION](https://arxiv.org/abs/2401.03568) and embed some article paragraphs.\n",
    "Since the article is too long, we first need to segment the article. Here, we use a sequential tokenizer without any optimization to split the article into small text chunks of 150 characters each.\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "f50be353df4cc168"
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "outputs": [
    {
     "data": {
      "text/plain": "['\\nMultimodal Agent AI systems have many applications. In addition to interactive AI, grounded multimodal models could help drive content generation for',\n ' bots and AI agents, and assist in productivity applications, helping to re-play, paraphrase, action prediction or synthesize 3D or 2D scenario. Funda',\n 'mental advances in agent AI help contribute towards these goals and many would benefit from a greater understanding of how to model embodied and empat',\n 'hetic in a simulate reality or a real world. Arguably many of these applications could have positive benefits.\\n\\nHowever, this technology could also be',\n ' used by bad actors. Agent AI systems that generate content can be used to manipulate or deceive people. Therefore, it is very important that this tec',\n 'hnology is developed in accordance with responsible AI guidelines. For example, explicitly communicating to users that content is generated by an AI s',\n 'ystem and providing the user with controls in order to customize such a system. It is possible the Agent AI could be used to develop new methods to de',\n 'tect manipulative content - partly because it is rich with hallucination performance of large foundation model - and thus help address another real wo',\n 'rld problem.\\n\\nFor examples, 1) in health topic, ethical deployment of LLM and VLM agents, especially in sensitive domains like healthcare, is paramoun',\n 't. AI agents trained on biased data could potentially worsen health disparities by providing inaccurate diagnoses for underrepresented groups. Moreove',\n 'r, the handling of sensitive patient data by AI agents raises significant privacy and confidentiality concerns. 2) In the gaming industry, AI agents c',\n 'ould transform the role of developers, shifting their focus from scripting non-player characters to refining agent learning processes. Similarly, adap',\n 'tive robotic systems could redefine manufacturing roles, necessitating new skill sets rather than replacing human workers. Navigating these transition',\n 's responsibly is vital to minimize potential socio-economic disruptions.\\n\\nFurthermore, the agent AI focuses on learning collaboration policy in simula',\n 'tion and there is some risk if directly applying the policy to the real world due to the distribution shift. Robust testing and continual safety monit',\n 'oring mechanisms should be put in place to minimize risks of unpredictable behaviors in real-world scenarios. Our “VideoAnalytica\" dataset is collecte',\n 'd from the Internet and considering which is not a fully representative source, so we already go through-ed the ethical review and legal process from ',\n 'both Microsoft and University Washington. Be that as it may, we also need to understand biases that might exist in this corpus. Data distributions can',\n ' be characterized in many ways. In this workshop, we have captured how the agent level distribution in our dataset is different from other existing da',\n 'tasets. However, there is much more than could be included in a single dataset or workshop. We would argue that there is a need for more approaches or',\n ' discussion linked to real tasks or topics and that by making these data or system available.\\n\\nWe will dedicate a segment of our project to discussing',\n ' these ethical issues, exploring potential mitigation strategies, and deploying a responsible multi-modal AI agent. We hope to help more researchers a',\n 'nswer these questions together via this paper.\\n\\n']"
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import faiss\n",
    "\n",
    "embedding_text = \"\"\"\n",
    "Multimodal Agent AI systems have many applications. In addition to interactive AI, grounded multimodal models could help drive content generation for bots and AI agents, and assist in productivity applications, helping to re-play, paraphrase, action prediction or synthesize 3D or 2D scenario. Fundamental advances in agent AI help contribute towards these goals and many would benefit from a greater understanding of how to model embodied and empathetic in a simulate reality or a real world. Arguably many of these applications could have positive benefits.\n",
    "\n",
    "However, this technology could also be used by bad actors. Agent AI systems that generate content can be used to manipulate or deceive people. Therefore, it is very important that this technology is developed in accordance with responsible AI guidelines. For example, explicitly communicating to users that content is generated by an AI system and providing the user with controls in order to customize such a system. It is possible the Agent AI could be used to develop new methods to detect manipulative content - partly because it is rich with hallucination performance of large foundation model - and thus help address another real world problem.\n",
    "\n",
    "For examples, 1) in health topic, ethical deployment of LLM and VLM agents, especially in sensitive domains like healthcare, is paramount. AI agents trained on biased data could potentially worsen health disparities by providing inaccurate diagnoses for underrepresented groups. Moreover, the handling of sensitive patient data by AI agents raises significant privacy and confidentiality concerns. 2) In the gaming industry, AI agents could transform the role of developers, shifting their focus from scripting non-player characters to refining agent learning processes. Similarly, adaptive robotic systems could redefine manufacturing roles, necessitating new skill sets rather than replacing human workers. Navigating these transitions responsibly is vital to minimize potential socio-economic disruptions.\n",
    "\n",
    "Furthermore, the agent AI focuses on learning collaboration policy in simulation and there is some risk if directly applying the policy to the real world due to the distribution shift. Robust testing and continual safety monitoring mechanisms should be put in place to minimize risks of unpredictable behaviors in real-world scenarios. Our “VideoAnalytica\" dataset is collected from the Internet and considering which is not a fully representative source, so we already go through-ed the ethical review and legal process from both Microsoft and University Washington. Be that as it may, we also need to understand biases that might exist in this corpus. Data distributions can be characterized in many ways. In this workshop, we have captured how the agent level distribution in our dataset is different from other existing datasets. However, there is much more than could be included in a single dataset or workshop. We would argue that there is a need for more approaches or discussion linked to real tasks or topics and that by making these data or system available.\n",
    "\n",
    "We will dedicate a segment of our project to discussing these ethical issues, exploring potential mitigation strategies, and deploying a responsible multi-modal AI agent. We hope to help more researchers answer these questions together via this paper.\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "chunk_size = 150\n",
    "chunks = [embedding_text[i:i + chunk_size] for i in range(0, len(embedding_text), chunk_size)]\n",
    "chunks"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-01-28T04:53:47.458376Z",
     "start_time": "2024-01-28T04:53:47.332607Z"
    }
   },
   "id": "a08f0a6414229633"
  },
  {
   "cell_type": "markdown",
   "source": [
    "接着，我们将这些小文本块进行 Embedding，得到一个 1024 维的向量。然后，我们将这些向量存入到一个向量数据库中，以便后续进行检索。\n",
    "\n",
    "Next, we Embedding these small text blocks to obtain a 1024-dimensional vector. We then store these vectors into a vector database for subsequent retrieval.\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "7afae1cb788176f6"
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "outputs": [
    {
     "data": {
      "text/plain": "23"
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import normalize\n",
    "import numpy as np\n",
    "import faiss\n",
    "\n",
    "embeddings = []\n",
    "for chunk in chunks:\n",
    "    response = client.embeddings.create(\n",
    "        model=\"embedding-2\",\n",
    "        input=chunk,\n",
    "    )\n",
    "    embeddings.append(response.data[0].embedding)\n",
    "normalized_embeddings = normalize(np.array(embeddings).astype('float32'))\n",
    "d = 1024\n",
    "index = faiss.IndexFlatIP(d)\n",
    "index.add(normalized_embeddings)\n",
    "\n",
    "n_vectors = index.ntotal\n",
    "\n",
    "n_vectors"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-01-28T04:54:02.573792Z",
     "start_time": "2024-01-28T04:53:47.455476Z"
    }
   },
   "id": "36de30ea532f5804"
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Search\n",
    "\n",
    "我们可以使用向量数据库进行检索。下面代码实现了一个名为`match_text`的函数，其目的是在一个文本集合中找到与给定输入文本最相似的文本块。\n",
    "其中 `k`是要返回的相似文本块的数量。\n",
    "\n",
    "We can use the vector database for retrieval. The following code implements a function called `match_text`, whose purpose is to find the text block in a text collection that is most similar to the given input text.\n",
    "where `k` is the number of similar text blocks to be returned."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "2518eb00681aea61"
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import normalize\n",
    "\n",
    "\n",
    "def match_text(input_text, index, chunks, k=2):\n",
    "    k = min(k, len(chunks))\n",
    "\n",
    "    response = client.embeddings.create(\n",
    "        model=\"embedding-2\",\n",
    "        input=input_text,\n",
    "    )\n",
    "    input_embedding = response.data[0].embedding\n",
    "    input_embedding = normalize(np.array([input_embedding]).astype('float32'))\n",
    "\n",
    "    distances, indices = index.search(input_embedding, k)\n",
    "\n",
    "    for i, idx in enumerate(indices[0]):\n",
    "        print(f\"similarity: {distances[0][i]:.4f}\\nmatching text: \\n{chunks[idx]}\\n\")\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-01-28T04:54:02.574230Z",
     "start_time": "2024-01-28T04:54:02.570496Z"
    }
   },
   "id": "ab40ecce590db788"
  },
  {
   "cell_type": "markdown",
   "source": [
    "我们可以使用这个函数来检索一些文本。例如，我们可以检索一些与“VideoAnalytica dataset”最相似的文本块。\n",
    "\n",
    "We can use this function to retrieve some text. For example, we can retrieve some text blocks that are most similar to \"VideoAnalytica dataset\"."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "2c03255c947074a3"
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "similarity: 0.5244\n",
      "matching text: \n",
      "oring mechanisms should be put in place to minimize risks of unpredictable behaviors in real-world scenarios. Our “VideoAnalytica\" dataset is collecte\n",
      "\n",
      "similarity: 0.4166\n",
      "matching text: \n",
      "tasets. However, there is much more than could be included in a single dataset or workshop. We would argue that there is a need for more approaches or\n"
     ]
    }
   ],
   "source": [
    "input_text = \"VideoAnalytica dataset\"\n",
    "\n",
    "matched_texts = match_text(input_text=input_text, index=index, chunks=chunks, k=2)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-01-28T04:54:03.258173Z",
     "start_time": "2024-01-28T04:54:02.573968Z"
    }
   },
   "id": "7d199339bf45c182"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
